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THE  OPTIMUM  ALLOCATION  OF  WEAPONS  TO  TARGETS: 

A  DYNAMIC  PROGRAMMING  APPROACH 

ABSTRACT 

The  Optimality  Principle  of  Dynamic  Programming  is  used  to  determine 
the  optimum  solution  of  a  weapons  allocation  problem  for  the  case  involving 
a  total  weight  constraint  on  the  amount  of  ammunition  employed. 
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INTRODUCTION 


In  many  military  combat  situations  one  is  confronted  with  the  task  of 
determining  how  the  available  resources  of  weapons  (or  actions)  may  be 
allocated  to  a  given  target  complex  so  as  to  minimize  the  expected  number 
or  the  "worth"  of  surviving  targets.  Since  each  weapon  (or  action)  has 


8,880013.1)0(1  with  it  8  certain  probability  of  destruction  which  is  dependent 
upon  the  type  or  class  of  targets  against  which  the  weapon  is  directed, 
then  one  will  need,  in  general,  different  amounts  of  each  weapon  available 
in  inventory.  Thus,  the  planning  stage  of  such  an  activity  requires  (among 
other  things)  the  determination  of  the  number  of  weapons  of  a  certain  type 
which  will  be  needed  to  carry  out  the  given  objective.  That  is,  given  a 
complex  of  m  targets  and  n  types  of  weapons  that  may  be  directed  against 
the  m-targets  how  often  should  weapon  i  be  directed  against  target  j  so 
as  to  accomplish  the  mission?  If,  after  analysis,  it  is  determined  that 
weapon  i  should  be  directed  against  target  j,  K,  times,  then  it  is  clear 


that  one  needs  K  = 


2  kj 


rounds  of  type  i  on  hand. 


However,  in  practice. 


it  may  not  be  possible  to  fulfill  the  demands  for  weapon  i  due  to  restrictions 
placed  upon  our  resources.  That  is,  due  to  limited  production,  size  and 
weight  restrictions,  shipping,  etc.,  it  may  not  be  possible  to  achieve  the 
level  of  damage  desired  or  to  totally  destroy  the  target  complex  (which  would 
probably  be  possible  with  unlimited  resources)  so  that  our  problem  reduces 
to  one  of  doing  the  best  possible  job  with  the  available  resources.  That 
is,  we  wish  to  minimize  the  expected  "worth"  of  the  surviving  targets  where 
we  have  limited  resources  available.  In  general,  this  problem  may  be  de¬ 


scribed  as  a  multi -dimensional  allocation  process,  but  with  the  particular 
constraint  that  will  be  used  in  the  problem  to  be  described  in  the  following 
pages,  it  will  be  possible  to  reduce  this  problem  to  a  sequence  of  simpler 


problems . 


THE  FORMULATION  OF  THE  PROBLEM 


The  problem  that  we  shall  discuss  is  a  weapon  allocation  problem  related 
to  problems  of  current  interest  to  the  Weapon  Systems  Laboratory  of  BRL.  We 
shall  use  the  following  notations  and  definitions: 
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•i.v 


*ij! 


a . : 

J 


the  conditional  probability  that  target  T  .  is  destroyed  by 

J 

action  A. ,  given  that  the  target  survives  all  other  actions 

i'  -  w 

1  -  p.  . 

1,1 

the  worth  of  target  T  . 

J 


w  :  the  weight  of  a  T,missileu  from  action  A. 


x.  . ; 
ij 


the  number  of  times  that  action  A^  is  directed  against  target  T. 


An  Interpretation  of  such  words  as  action,  worth,  missile,  etc.,  may 
be  provided  by  thinking  of  an  action  as  the  firing  of  a  particular  type  of 
projectile,  "missile",  shell,  etc.,  against  a  target.  The  weight  would  be 
the  weight  of  the  given  shell.  Since  each  type  of  shell  may  have  different 
blast  properties,  fragmentation  characteristics  or  different  type  warheads 
one  may  consider  action  A  as  the  firing  of  shell  of  type  i,  i.e.,  each  shell 


in  this  group  has  similar  properties 


me  x.  .  would  be  the  number  of  times 
i  J 

that  shells  of  type  i  (action  A  )  are  directed  against  target  T.. 

i  J 

The  worth  of  a  target  is  a  measure  of  the  importance  the  offense  has 

assigned  to  that  particular  target.  The  constraint  on  our  resources  will  be 

a  total  weight  constraint,  W.  That  is,  if  is  the  weight  of  all  projectiles 

n 

included  in  A.  and  we  have  n  actions  then  W  =  £  K.  where  W  is  a  fixed 
1  i=l  1 

preassigned  number.  The  problem  is:  given  n  actions  and  m  targets;  and 

a  fixed  total  weight,  W,  determine  the  set  of  values  {  x.  .  |  which  minimizes 

V  I  J  J 

the  expected  worth  of  the  surviving  targets. 

The  expected  worth  of  the  surviving  targets  is  a  function  of  nm  variables 
and  is  given  by 


(i) 


F(x1,x2, 


m 

n 

=  2  a. 

7T 

J 

i — 1 

II 

•H 

H 

tJL 

X.  . 


where 


X  =  (x  . ,x  . . . ,x  ) 
n  nl  n2  nm 
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The  problem  then  is  to  select  the  set  of  m n  numbers  x^  i  =  i,2,...,n; 
j  =  1,2, ... ,m  (or  the  n  m- dimensional  vectors)  which  gives  the  minimum 
value  of  F  subject  to  the  conditions: 


(a) 

0  <  q,  . 

U 

<  1  for  all  i  and  j 

00 

X.  . 

IJ 

>  0  ;  in  particular  x^  is  to  be  a  non-negative 

integer. 

JLJL 

w 

111 

and 

(c) 

2  K 

y  x. .)  <  w 

^  ij  - 

i — i 

.a 

C_i. 

II 

H 

We  shall  call  x. .  an  allocation  (an  allocation  of  a  "missile"  from 
ij 

action  to  the  jth  target)  and  a  sequence  of  allocations 

x_, ,  • . . ,  x_  , . . . ,  x  x  which  satisfies  the  constraints  (b)  and  (c) 

21  2m  nl7  nm 

we  shall  refer  to  as  a  policy.  We  can  then  speak  of  finding  an  optimal 
policy,  that  is,  a  policy  which  results  in  a  minimum  value  of  F. 

DYNAMIC  PROGRAMMING  FORMULATION 


■\  r*  1 1  <1  4~  -?  Vi 

u  iuu 


r~<  1 1V1  -i 


and  the  dimension  involved  quickly  convinces  one  that  naive  approaches 
such  as  enumeration  of  policies  or  the  use  of  classical  methods  to  obtain 
the  minimum  value  of  F  are  nob  feasible  because  of  the  unreasonable  co  input  - 
ing  times  involved  and  the  complexity  of  the  problem.  For  example,  if 
n  =  m  =  4,  and  each  of  the  x. .  are  permitted  to  run  over  ten  values,  the 

—  u 

computing  time  involved  in  enumeration  of  F  would  exceed  by  several  fold  the 
life  time  of  any  modern  day  computing  machine. 


In  order  to  circumvent  this  dimensional  difficulty,  we  will  reformulate 


U-'U  G  n  ml  1 1  •) — ?  _  r*  4 -  o  rrrs  o  r-  nnrl  Q-nr\l  ir  4-V»  O  4*  *1  nilDO  r* 

UI1-LD  U_L.Ciii  ao  a.  lUUJ.U±-0  uagt  wixv.  \SJ.  j . 


programming  to  obtain  a  feasible  computational  scheme.  Each  stage  of  the 

process  will  consist  of  determining  a  value  for  a  specific  allocation,  xn.  .. 

-1*  o 

Our  goal  shall  be  the  reduction  of  this  mn  dimensional  problem  to  a  sequence 
of  one -dimensional  problems.  Then  at  each  stage  of  the  process  it  will  be 
possible  to  determine  an  allocation  from  a  search  over  a  one -dimensional 
space.  To  attain  this  simplification  we  imbed  this  particular  problem  within 
a  family  of  analogous  problems,  that  is,  instead  of  considering  a  particular 
weight  of  resources  and  a  fixed  number  of  actions  and  targets,  we  consider 
the  entire  family  of  such  problems  where  W  may  assume  any  positive  value  and 
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n  and  m  may  be  any  natural  number.  This  approach,  as  we  will  see,  has  many 
computational  advantages  and  furthermore,  we  shall  be  able  to  obtain  vital 
information  about  the  change  in  optimal  policies  as  the  basic  parameters,  W, 
n  and  m  change,  i.e.,  we  can  assess  the  effect  on  the  optimal  policy  of  add¬ 
ing  additional  targets  and  additional  actions  or  of  varying  the  total  weight 
constraint. 


In  the  analysis  which  follows,  the  reduction  in  dimensions  takes  place 
in  two  steps,  the  two  steps  come  about  through  the  analysis  of  the  structure 
of  the  problem.  Our  problem  is  to  determine  the  policy  jx.^,  . . .  ,x^; 

xpl  ,xpp, ...  ,xp7n; ...  ;xnl ,...  ,x^m  \  which  minimizes  the  expression  F,  i.e., 

m  x,  . 


Min 


nl* 


2 

j=l 


»  c 

7 r 

i=l 


ij 


subject  to  constraints (a),  (b)  and  (c).  We  can  write  this  as  follows: 


n 


Min  an 

i  i 

tAii/ 


il 


V  qii  r ..  i 
i=l 


+  Min 
{Xi2 1 


a. 


n 

7 T 

i=l 


Xi2 


q._  +...  +  Min  a  -rr  Q. 

i2  f  1  m  'I  im 


im 


lxi4 


i=l 


The  jth  term  in  this  sum  is  a  minimization  over  the  set  of  values  ^x^., 
x^.,...,x  .  ^  and  since  we  can  write  the  constraint  (c)  as 

>>  ”|  7  7  y~i  -i  1 

,  J  J-L.l  + 


(c)  2j  (V.  Z 
i=l 


m 

n 

n 

m 

r' 

r-» 

z 

>i 

X.  .)  =  2j  v.  X.  . 

1  l  .  .  1  i  ^ 

1=1  ~  ° 

+  2, 

i=l 

(v_.  Zj 

*  k=l 
/ 

x.,  )  <  W 

lk  — 

Kf  J 


The  x .  . ,  i  =  1,  ....  n,  involved  in  the  jth  summand  must  satisfy  a 
i  J 


constraint  of  the  form 


n 

V 

Z- 

i=l 


w.  x.  .  =  z  <  W. 
i  ij 


Thus  the  j-th  term  in  the  sum  is  a  function  of  the  number,  k,  of  actions 

directed  against  target  T.  and  the  combined  weight,  z.,  of  the  "missiles1', 

J  d 

which  are  allocated  to  T..  We  denote  the  minimum  value  of  the  j-th  term 

low  -PJ  f  rv  \  n  {=> 
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.,j/_  \  /  xl.l  x2.i  *k.ix 

W  =  .,x,  .\  ^lj  ^2  j  ...  q,  . 

l  xj  J  U£j 


where 


k 

kA  V'  _  ,  »  ~  ^  TT 

u  a  a.  .w  .  =  z.  .  %  w 

*  ,-_1  1J1  J  - 


The  minimization  involved  in  the  above  equation  can  be  accomplished  in  k  one¬ 
dimensional  minimization  processes  by  employing  Bellman's  simple  but  powerful 
Principle  of  Optimality.^  We  then  have 


(2)  f,^  (z  .)  =  Min  (a.*^  f.^  .  (z  .  -  w  x  )  1  :  k  =  > . n 

&  '  3'  '^3  i-l  '  j  k  kj'" 

Kj 


where  x_  .  can  assume  the  values  0.1.2 . I  z  /w  1  .  Thp  s-vTOhnl  T  vl  in 

'icj  '  -  [7  kj  ’  - " - L^J  ~ 

defined  to  be  the  greatest  integer  which  does  not  exceed  y.  When  k  =  1, 


(3) 


i  x-i  • 

f,  (z  .)  =  Min  (a  q, 
x  •  J'  -  J"iJ  * 

XJ 


where  x^  £  {°’xx-  [*A]}  ,  thus 
(4) 


1  .  .  fV^l 

q  CZjJ  =  a^qj-.0  for  all  j  =  1,2,..., m. 


Equations  (2)  and  (3)  hold  for  all  j  =  l,2,...,m;  however,  it  should  be 


noted  that  although  z  .  is  unknown,  we  do  know  that 

J 


0<z+z_+...+z  =  z  <  W,  and  hence  that  0  <  z  .  <  W. 

—  L  d  m  —  -J- 

^.^,4  J3  ~  _ n _ 

fei-LU.  Ui  Zj  VfcLLUeS. 

z.)j-  and  the  corresponding 

optimal  allocations  ( x,.,(z  J-).  .  The  former  set  of  values  must  be  stored  in 

l  *-«J  J  J 

the  computer  memory  in  order  to  determine  f£"£  .  At  this  stage  of  the  process 

we  have  obtained  some  vital  information  as  to  the  effect  on  the  optimal  policy 
with  respect  to  the  j-th  target,  resulting  from  a  change  of  total  weight 
allocated  to  the  j-th  target  and  also  the  effect  resulting  from  a  change  in 
the  number  of  actions  directed  against  the  j-th  target. 

Finally  we  define  a  function  g  depending  on  the  total  weight  allocated 
to  a  total  of  N  targets,  and  the  number,  N,  of  targets  against  which  n  actions 
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rTTU  4  n  *4  wml  4  +Vi  n  4-  t.t/n  vmi  i  n4-  4*  nl\i  iT  n4« 

luio  jJapd.  ico  0X10.0  tvc:  iuud  o  oauuiabc  _  %  .  /  _ 

J  r  f  A  \ 

That  is,  we  obtain  from  eq.  (2)  the  sequence  |f^;  ( 


t„  \  _ 

v  kj  .  ;  uvci 

j 


are  directed  and  where  an  optimal  allocations  of  weights,  z,  to  each  of 


4*  q  rrc*  4-  e?  H"1  4  c?  noo4 

Uilt.  UUlgj'w  (JO  J.  .  . 

J 


->  ■+*  o  "v* era +-  e?  1  I 1  ~i  c  noor)  f"PVi  Q  4"  4  c 

,1^  UUl  kJ  _L  .  -J-  HU-  ^ 


(5)  e„(z,n)  =,  Mi11 

y  zpj  1 


\  zi ,z^ 


.(^(O  +  ff(z„)  +  ...  +  f?  (z„))  ; 


z-l- 


iJ.  £_ 


■LI  1M 


N  =  1,2,. 


>  N 

•  in* 


TA7>ior*o  *7  -4-7  -4- 

“1  '  “2  ' 


4-7  —  7 

•  “u 


w. 


When  N  =  m  and  we  have  obtained  the  optimal  allocation  of  weights -£z^, z^, ...  ,z 

4  o  -f  "ho  oomionoo  /  7  \  wVi  4  nln  m4n4m4  7PC  PYnvpqc4nn  ( ^\  “HVio-n  4-Vu=>  -rrv*/~», 

X  •  ^  Uiiv^  tJU  V^uuus^  ^  *-»  ^  J'  nil  aixuxiux  »-?  vxa  w  x  ^  ^  -tvi4  \  */  J  ^llv^U  ^X 

complete  and  the  solution  to  our  problem  is  obtained.  Again,  employing 
Bellmans  Optimality  Principle  equation  (5)  becomes 


m 


/  \  ..  t  _  ..  \  /JJ  /_  X  .  -  /_  _  \  \  .  tt  r\  -7 

\V)  ^z,n;  =  Min  ^In  VZN;  -1- 

7 

“N 

and  where  0  <  z  <  z.  For  N  =  1,  g^(z,n)  becomes 


(7)  gj_(z,n)  =  F(z). 

Since  equations  (6)  and  (7)  are  true  for  all  n,  m,  and  z  we  can  assess  the 

r  ■» 

effect  on  the  optimal  policy  |  x1  ^ .  ,x.)  . . .  ;x^  , . .  .x^m  f  which  results  from 

a  variation  in  any  one  of  the  parameters.  However  if  we  are  interested, 
always,  in  a  particular  value  of  n,  say,  n  =  k,  then  considerable  reduction 
in  computation  is  achieved  since  one  needs  to  compute  g^(z,k),  only,  instead 
of  gII(z,l),...,gN(z,k),...gN(z,n). 

COMPUTATIONAL  PROCEDURE 


The  dynamic  programming  formulation  automatically  imbeds  the  original 
problem  within  a  family  of  analogous  problems  in  which  the  basic  parameters 
n,  m  and  W  assume  sets  of  values  which  permit  us  to  obtain,  in  the  course 
of  the  computation,  the  solution  to  a  variety  of  sub-problems.  Thus,  great 
savings  in  time  and  labor  can  be  obtained  by  careful  analysis  of  the  class 
of  particular  problems  which  are  of  interest.  This  is  of  great  importance 
because  of  the  very  structure  of  the  process,  i.e.,  we  use  the  information 
obtained  from  sub-problems  of  the  original  to  obtain  the  solution  of  the 
original  problem.  For  example,  consider  the  following  two  sets  of  actions, 
targets  and  weights: 
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Then,  it  is  clear  that  we  should  obtain  the  solution  to  problem  II,  since 
by  adding  actions  and  targets,  according  to  subscript  ordeq  we  have  solved 
problem  I  in  an  intermediate  stage  (provided  we  include  VL  in  the  grid  of  z 
values).  This  results  from  the  recursive  relationships  involved  in  equations 
2,3,6  and  7* 

Now  let  us  examine  the  computational  procedure,  in  detail,  and  determine 

what  information  is  needed  to  start  the  computation.  The  input  information 

that  is  required  is  the  knowledge  of  the  values  of  n,  m,  W  and  the  values 

of  the  q. .  *s.  Again  if  we  desire  solutions  for  a  set  of  values  of  n,  m  and  W, 
i  J 

the  proper  indexing  of  the  actions  and  the  targets  is  essential  since  we 
wish  to  determine  solutions  for  as  wide  a  subclass  of  problems  as  possible 
since  we  can  use  these  solutions  to  obtain  solutions  for  a  larger  class  of 
problems  of  interest.  We  shall  illustrate  this,  later,  by  a  numerical  example. 

Given  the  above  information  we  select  the  largest  W  for  which,  we  desire 

a  solution.  Now  equations  2  and  5  involve  weights  z  .  where  0  <  z  .  <  W,  that 

is,  we  compute  f^(z  .)  for  a  set  of  z.fs.  The  choice  of  the  values  of  z.*s  that 

J  J  J 

we  must  select  is  restricted  by  available  computer  memory,  time  and  cost.  If 

we  are  interested  in  a  set  of  Wfs  then  these  values  should  be  included  in  the 
set  of  z.*s  and  the  remaining  values  of  z.fs  are  to  be  chosen  in  such  a  way 
as  to  eliminate  the  need  for  sophisticated  methods  of  interpolation,  if  this 
is  possible,  and  the  grid  should  be  chosen  coarse  enough  so  that  time  require¬ 
ments  are  not  excessive.  Unfortunately,  there  seems  to  be  no  a  priori  method 
of  optimizing  this  selection  of  grid  points,  rather,  one  must  rely  on  experience, 
ingenuity  or  divine  guidance. 

Once  the  grid  of  z  .  values  has  been  chosen  the  calculation  begins  with 
<3 

equation  (4).  That  is,  we  compute  and  store  (and  print)  the  following  values 
for  each  target  T . : 
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r 


Table  I 


fi  (zh  ;  xij(zi) 
H  (Z0)  >  X1  lZo) 


ljv  2' 


<  : 


fi  (zP  ’  xij(zi} 

where  xn  ,(z_)  is  the  allocation  of  weapon  of  type  A  to  the  j-th  target, 
i  r  _L 

given  a  weight  constraint  z^ .  We  compute  the  table  for  each  j  separately, 
since  in  general,  the  memory  capacity  of  the  computing  machine  may  not  be 

great  enough  to  accommodate  the  storage  of  the  2m  x  I  values  of  ffhz  )  ; 

^  Li  ^ 

x  (z  )  ?*  ,  j  =  1, . .  .  ,m  and  r  =  1,  which  would  be  required  for  the 

J-j  r  - 

simultaneous  computation  of  the  m  targets.  After  the  set  of  values  Jfd(z  )  ; 

X-j  .(z^)^;  r  =  1,2,...,  £  is  obtained  for  the  j-th  target  then  the  set  of 

values  Jflz  ) ,  x_,(z  )V  is  computed  from  equation  (3).  The  calculation  of 
r  dy  r  J  "  ' 

f,i(z.)  involves,  of  course,  a  knowledge  of  f!r(z  )  where  z><,  z  .  This  works 
£_1  -L  K  K  r 

to  our  advantage  since  when  Xp.(z^)  is  determined  then  xn  .  can  be  determined 


from  the  set  of  values  Jf  ^(z 


-  x„  ,w„)  :  x-  ,  (z 
d-  '  1J  ' 


-  X2jW2l  ~y  in~erPolati°“ 


\  1  r  zj  a’  '  lj'  r  2j  2J  - 

of  Table  I,  thus  we  obtain  the  optimal  policy  Jx^j,  X2jr  ^or  target 

corresponding  to  the  two  actions  and  the  weight"  restriction  z  .  We  continue 

this  calculation  (eq.  2)  until  the  desired  number;  n,  of  actions  is  obtained. 

At  this  stage  of  the  calculation  we  have  the  following  sets  of  values  for  the 

j-th  target:  (z  )1  and  the  allocations  J xn  x^.,...,x  .\where 

L  n  v  r'J  \  lj’  2.j’  ’  njj 

X  .  =  x  .(z  )  and  x  .  .  =  x  .  .(z  -  >  x  .w  ) ;  k=0,l,2, . . . ,n-l. 

nj  nj  r7  n-k,j  n-k,jv  r  4-u  sj  syj  9  ’  ’  ’ 

s=n-k+l 

We  now  have  the  necessary  information  to  compute  the  functions  defined  by 
equations  5  -  1>  in  fact,  at  this  stage  we  have  g^(z,n)  since  this  is  f^(z). 
Thus,  we  start  with  equation  (6)  and  compute  gQ(z,n),  i.e., 


(7) 


r  2 

g2(z,n)  -  Min  ^f  (z2)  +  gx(z  -  Zg 

z  L 11 


for  0  z  W, 


In  computing  gg(z,n),  we  are  again  confronted  with  the  problem  of  our 
choice  of  grid  values  for  z.  Now,  zp  represents  an  allocation  of  weight 
resulting  from  the  assignment  of  n  actions  against  target  Tg,  thus  it  is 


14 


clear,  that  the  values  of  should  run  over  values  which  are  linear 
combinations  (with  non-negative  integral  coefficients)  of  the  weights  w^, 
Wg,...,  and  only  those  linear  combinations  of  the  weights  which  are 
less  than  z.  Any  other  assignment  of  weight  to  target  2  would  result  in 
a  waste  of  weapons  since  we  only  consider  the  case  where  the  are  non¬ 
negative  integers.  In  general,  it  will  not  be  possible  to  include  all  such 
linear  combinations  of  the  weights  because  of  restricted  machine  memory 
and  excessive  computing  times. 


Once  the  value  of  z^,  which  minimizes  the  expression  in  (6)  above,  has 

been  determined,  then  we  can  determine  the  allocation  to  target  2  (and  target  l) 

2 

from  our  knowledge  of  f  (z)  and  the  corresponding  allocations  which  were 


n  2  (  > 

determined  previously,  i.e.,  we  have  g^(z,n);  f^z^)  |  XJ2’  X22*  •••  >  xn2  j 

and  also  g^(z  -  z^n)  =  f^(z  -  )  from  which  we  obtain  ^x^,  x21*  ...  ,  x^^y 

These  sets  of  xfs;  <(x10,  ...  ,  and  ^x1i;  xpi,„...  ,  x^^  represent  £he 

optimal  policy  for  the  case  where  we  have  n  actions,  2  targets  and  a  total  weight 
constraint  z.  We  continue  the  above  process  for  all  choices  of  z  and  then 
g^(z,n)  can  be  computed  in  the  same  fashion  obtaining  the  optimal  allocation  for 
the  case  of  3  targets,  n-actions  with  a  total  weight  constraint  z.  Thus  for 
similar  values  of  z  we  can  observe  the  change  in  optimum  policy  as  more  targets 
are  added  to  the  process  -  we  can  observe,  also,  the  change  in  optimum  policy  at 
any  stage  as  a  function  of  z.  The  above  procedure  is  continued  until  the  desired 
number  of  targets  has  been  included,  i.e.,  until  g^(z,n)  is  obtained.  A 
similar  calculation  is  made  for  each  n  that  is  desired. 


We  conclude  this  discussion  with  a  simple  numerical  example  which  we  hope 
illustrates  some  of  the  salient  features  of  this  approach  -  although  -  one 
which  eliminates  many  of  the  difficulties  which  may  arise  in  any  realistic 
problem. 
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A  NUMERICAL  EXAMPLE 


Ri irvnn^p*  -wr  hntrpi  n  =  2  actions.  A  . 

- jrj r ~  ^  *J-  *-  3_^  2 


A 

> 


T  ^  j_  _  /-\  /“\ 

jjt;  u  ci^  -  u.^, 


a0  =  0.3^  and.  =  0.5  and  let  us  assume  that  q^  =  0.1,  q^  =  0.4, 

q  =  0.6,  qpi  =  0.5,  qpp  =0.3  and  q^  =  0.2.  Also,  wn  =  2  lbs., 

w„  =  3  lbs.  and  W  =  10  lbs.  We  wish  to  minimize  equation  (l)  with  those 

values;  i.e.,  we  wish  to  determine  the  optimum  policy  /  x,  ..,x,  OJx  ,x  ,x  .x  1 

l  11  12>  13'  21’  22  23 J 

We  begin  the  calculation  with  equation  (4)  and  compute  the  value  of 


for  each  j  and  z  we 
K 


f^(z  ),  j  =  1,2,3  for  values  oi  z  =  u,i, 2, . . . , 10. 

J.  K  .  K. 

record  the  values  of  f^(z  )  and  the  value  of  x  .  which  gives  this  value  of 
•  IK  lj 

.  That  is,  we  construct  Tables  2  and  3: 


TABLE  2 


TABLE  3 


z.  \  .i 

KV 

i  1 

2 

1 

N 

£ 

]_ 

O 

£_ 

J 

0 

0.20 

0.30 

0.50 

0 

0 

0 

0 

1 

0.20 

o*  30 

0.50 

1 

0 

C\ 

r\ 

u 

r\ 

a 

0. 02 

0. 12 

0. 50 

2 

1 

1 

1 

3 

0.02 

0.12 

0.30 

3 

1 

1 

1 

4 

0.002 

0.048 

0.18 

4 

2 

2 

2 

£=; 

r\  nno 

r\  rili  Q 

r\  i  Q 

rr 

r\ 

U  •  UU£L 

\J  •  UH-U 

U  •  XU 

u 

d 

d 

d 

6 

0.0002 

0.0192 

0.108 

6 

3 

3 

3 

7 

0.0002 

0.0192 

0.108 

7 

3 

3 

3 

8 

0.00002 

0.00768 

0.0648 

8 

4 

4 

4 

9 

0.00002 

0.00768 

0.0648 

9 

4 

4 

4 

10 

0.000002 

0.003072 

0.03888 

10 

5 

5 

5 

The 

next  step  of  the  calculation  is 

to  compute  fp(z  ) 

.  2  K 

>  j  = 

1,2,3. 

Again,  we 

• 

record  the  value  of  x  which  gives  f^.  Since  H(z.  )  can  be  inter- 
preted  as  the  return  from  the  direction  of  2  actions  against  the  jth 


target,  then  we  must  obtain  also.  Now  x.^  is  obtained  from  table  3  and 
is  found  in  the  row  corresponding  to  z  -  3Xp.  and  in  the  jth  column.  Thus 
we  have  tables  4  and  5: 
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TABLE  4 


TABLE  5 

(x2J(zk>’  xl3(zk  -  3x^» 


2j' 


TP- 

1 

2 

3 

zkV 

l 

2 

3 

0 

0.2 

0.3 

0.5 

0 

(0,0) 

(0,0) 

(0,0) 

1 

0.2 

0.3 

0.5 

1 

(0,0) 

(0,0) 

(0,0) 

2 

0.02 

0.12 

0.3 

2 

(0,1) 

(0,1) 

(0,1) 

3 

0.02 

0.09 

0.1 

3 

(0,1) 

(1,0) 

(1,0) 

4 

0.002 

0.048 

0.1 

4 

(0,2) 

(0,2) 

(1,0) 

5 

0.002 

0.036 

0.06 

5 

(0,2) 

(1,1) 

(1,1) 

6 

0.0002 

0.0192 

0,020 

6 

(0,3) 

(0,3) 

(2,0) 

7 

0.0002 

0. 0144 

0.020 

7 

(0,3) 

(1,2) 

(2,0) 

8 

0.00002 

0.00768 

0.012 

8 

(0,4) 

(0,4) 

(2,1) 

Q 

0.00002 

0.00576 

o.oo4 

9 

(0,4) 

(1,3) 

(3,0) 

10 

0.000002 

0.003072 

0.004 

10 

(M 

(0.3) 

Jfool _ 

The  calculation  of  f^(z)  is  now  complete  and  we  are  now  able  to  compute 


n 


the  g  functions.  Since  g^(z,2)  =  f 2(z)  we  can  proceed  to  the  calculation  of 
gg(z,2)  (Equation  6).  Since  equation  6,  for  N  =2,  is 

g2(zk,n)  =  Minjf^Zg)  +  g1(zk  -  z2,n)} 

we  are  to  find  the  value  of  which  minimizes  the  expression  on  the  right 


for  the  given  z  . 

K. 


The  values:  z^  and  z, 

9  2  k 


z^  we  use  to  enter  the  Table  5 


z2t 


and  obtain  the  policy  {x22(zkt  x12(zk  -  3x22),  x2q(zk 
xi;L(zk  -  z2  -  3x21)  |  .  For  example ;  when  zk  =  6  lbs.,  g2(6,2)  =  0.068 

and  the  value  of  tftich  gives  this  value  of  g^(6,2)  is  z2  =  4  lbs.  Thus, 
entering  Table  5  at  row  z  =  4  lbs.,  we  find  in  the  j  =  2  column: 


X22  "  °’  X12 


=  2. 


In  the  same  table  in  row  zk  -  z2  =  6 


4=2  lbs.  We 

find  in  the  j  =  1  column  x^  =  0  and  xn 1  =  1.  Thus  the  optimal  policy  for 
two  targets,  two  actions  and  zk  =  6  lbs.  is  xqp.xq2.x2q>x22  }  =  {l, 2,0,0}  . 
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We  have,  now.  Table  6: 


TABLE  6 


zk 

{xll'x12> 

X21 

>XOC  \ 

^  / 

0 

0.5 

0,  0, 

0, 

0 

I 

0.5 

O 

•s 

O 

0, 

0 

2 

0.32 

1,0, 0,0 

or 

0 

*\ 

0 

•N 

1 — 1 

*\ 

0 

3 

0.29 

0,  0, 

0, 

1 

_l_ 

4 

T  1. 

u.  xq- 

0, 

0 

5 

O.ll 

1,  0, 

0, 

1 

6 

0.068 

1,  2, 

0, 

0 

7 

~!  -j 

r\ 

1 

U> 

_L 

0 

0 

0.0392 

l,  5, 

o, 

0 

9 

0.0344 

1,  2, 

0, 

1 

10 

0=0212 

0  X 

n 

W  y 

r\ 

V-/ 

Finally,  we  compute  g^(l0,2).  We  need  z  =  10  only  since  we  do  not 
need  to  compute  g^(z,2). 

Proceeding  as  above,  we  find  that  g,(l0,2)  =  O.I56  and  { x.,  ,x,  p,x.  , xp-. , 

x22,x23j-  =|l,  1,0, 0,l,l}  •  Examining  the  structure  of  the  above  calculation, 

it  is  readily  seen  that,  we  have  actually  solved  many  simpler  type  problems 

in  the  process.  For  example,  we  have  solved  the  problem  of  2  targets  (index 

1  and  2),  two  actions  and  a  total  weight  constraint  of  7  lbs.  The  optimal 

policy  for  this  problem  is  given  in  Table  6.  Furthermore,  if  one  is  interested 

in  the  same  targets  and  actions  but  with  a  total  weight  constraint  of,  say, 

9  lbs.,  then  g  (9,2)  must  be  computed.  For  this  example,  g,(9,2)  =  0.168 
^  r  ■*  ^ 

and  the  optimal  policy  is  1 1,2, 0,0, 0,1 ^  . 
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